Hello FCCAnalyses experts - I found that when submitting many jobs to HTCondor due to running over many processes and chunks, this would create a ton of directories, each used to run for a given job.

While I suppose one can create a run directory and simply submit jobs from there (actually, not sure if that would work since the condor script looks for setup.sh in your local_dir), which would populate that separated directory with lots of extra directories, I wanted to make things more organized and not dependent on the local directory you run from, so I propose with these changes to have each job's working directory be located inside the log_dir.

I'm not sure what other users prefer, but I wanted to share this code in case it's desirable/useful for the repo.

Files edited:

• python/run_analysis.py:
  • Moved definition and creation of log_dir outside of send_to_batch to ensure it can be used by create_condor_config so that in the condor script, job's are directed to the log_dir before creating a working directory.